1. Field of the Invention
The present invention relates to image search for searching for an image similar to a designated image based on the determination of similarities of an individual image and, more particularly, to an image search system for, even when feature of an image are represented by a different scheme, determining similarities of the image and searching for a similar image based on an image features amount according to each scheme, an image search method thereof and a storage medium which stores an image search program.
2. Description of the Related Art
With feature of an image made into data and represented by an image features, image search has been conventionally conducted by determining similarities of each image using the image features and searching for a similar image (e.g. from among accumulated images), for which devices and computer programs have been developed
These conventional image search techniques adopt a method of searching for a similar image mainly by using an feature which represents constituent of colors contained in an image (hereinafter referred to as color constituent features) to find similarities of the color constituent.
Here, the color constituent features denotes an image features expressing color constituent of the image, by a representative color in the image and a constituent rate of the same, etc. and it can be also expressed by a frequency rate of each color contained in the image, or one or a plurality of representative colors and its or their color constituent rates, etc.
The system using a color constituent features, however, has a problem that it is impossible to reflect structural feature of an image such as a shape and a positional relationship of objects in the image.
Here, one example of conventional techniques for reflecting structural feature that an image has is disclosed in Japanese Patent Laying-Open (Kokai) No. Heisei 11-316819. The conventional technique recited in Japanese Patent Laying-Open (Kokai) No. Heisei 11-316819 proposes a method of calculating a color constituent features contained in each block which is obtained by dividing an image into a plurality of blocks and using a set of values of the color constituent features of the respective blocks as a color distribution features for the determination and search of similarities. Since a color constituent features of each block is expressed in the color distribution features, structural feature of an image can be reflected in the determination of similarities.
Disclosed in Japanese Patent Laying-Open (Kokai) No. 2000-259832 as another conventional method is the image search system using, as an image features, the frequency distribution features which represents a color arrangement of an image by a frequency region.
Here, the frequency distribution features of an image denotes an image features which represents an energy of each band that is obtained by dividing a frequency distribution of a spectrum of a color (mean color) of an image into a plurality of bands and analyzing a frequency.
FIG. 11 is a block diagram showing a structure of a main part of the image search system disclosed in Japanese Patent Laying-Open (Kokai) No. 2000-259832. The conventional image search system includes an image features generation unit 102 for generating a frequency distribution features 103 from applied image data 101, a frequency distribution features storage unit 104 for storing a frequency distribution features of an image to be searched in advance, and a similarity calculation unit 106 for calculating a similarity 107 using the frequency distribution features 103 of the applied image and an feature 105 stored in the frequency distribution features storage unit 104. Here, the image features generation unit 102 includes a reduced image generation unit 110 for generating a reduced image 111 from the image data 101, a frequency analysis unit 112 for conducting frequency analyses of the generated reduced image 111, and a DC component and partial AC component extraction unit 114 for extracting a DC component and partial AC component as the frequency distribution features 103 from among the frequency analysis unit 112 and an orthogonal conversion coefficient 113.
As introduced in “Text of WD 4.0 of MPEG-7 Visual” (ISO/IEC JTC 1/SC29/WG11 N3522), one example for color distribution features could be a combination of the Grid Layout Descriptor and any of the Dominant Color, Color Structure and Scalable Color, and for frequency distribution features the Color Layout Descriptor is a typical example.
As described in the foregoing, among examples of an image features are a color constituent features, a color distribution features and a frequency distribution features, use of which realizes determination of similarities of an image and image search.
Other conventional technique is disclosed in Japanese Patent Laying-Open (Kokai) No. Heisei 09-237343 proposes an image search system which in combination uses, a color histogram as a color constituent features among the above-described image features amounts, and an orthogonal conversion coefficient as a frequency distribution features.
FIG. 12 is a block diagram showing a main part of the image search system disclosed in Japanese Patent Laying-Open (Kokai) No. Heisei 09-237343. The conventional image search system includes an input device 201 for receiving input of an image features, an image features storage unit 202 for storing an image features and a similarity calculation unit 203 for calculating a similarity between the image features applied through the input device 201 and the image features stored in the image features storage unit 202.
The image features storage unit 202 includes an orthogonal conversion coefficient storage unit 204 for storing a frequency distribution features of an image and a color histogram storage unit 205 for storing color constituent information.
The similarity calculation unit 203 includes an image features determination unit 206, a color histogram similarity calculation unit 207 and an orthogonal conversion coefficient similarity calculation unit 208. The image features determination unit 206 determines whether an image features 209 applied through the input device 201 represents color histogram information or an orthogonal conversion coefficient. When the image features determination unit 206 determines that the amount represents color histogram information, the color histogram similarity calculation unit 207 calculates a similarity 212 between an image features 210 composed of color histogram information which is output from the image features determination unit 206 and color histogram information 211 read from the color histogram storage unit 205. The orthogonal conversion coefficient similarity calculation unit 208 calculates a similarity 215 between an image features 213 composed of orthogonal conversion coefficients which is output from the image features determination unit 206 and an orthogonal conversion coefficient 214 read from the orthogonal conversion coefficient storage unit 204 when the image features determination unit 206 determines that the amount represents an orthogonal conversion coefficient.
Since for the system to conduct image search when a plurality of kinds of image features exist, it requires an image features amount data base and a similarity calculation means corresponding to an feature of an inquiry image, as many similarity calculation means and image features amount data bases as the number of kinds of image features should be provided, which makes the system complicated and manufacturing costs higher.
In order to solve the problem, a conventional image search system which realizes image search by a simple device structure even when image search is conducted using both a color constituent features and a frequency distribution features is proposed.
In the image search system, in a case where a device is internally provided with only a frequency distribution similarity calculation means, when an feature of an inquiry image or an image to be searched is a color constituent features, a mean color is calculated from the color constituent features and converted into a frequency distribution features to conduct similarity calculation based on the frequency distribution features.
On the other hand, in a case where the device is internally provided with only a similarity calculation means for a color constituent features, when an feature of an inquiry image or an image to be searched is a frequency distribution features, the image is restored by inversely converting the frequency distribution features and then color constituent information of the restored image is extracted to conduct similarity calculation based on the obtained color distribution features.
In a case where the system is provided with only a similarity calculation means for a frequency distribution features, when an feature of an inquiry image or an image to be searched is a color constituent features, a mean color is calculated from the color constituent features and the mean color is converted to a DC component of a frequency distribution features, thereby conducting similarity calculation based on the obtained frequency distribution features.
As a result, determination and search of similarities of an image whose feature is represented as a color constituent features or a frequency distribution features can be executed only by the provision of a similarity calculation means for either one of the kinds.
However, no method of alternately converting an image features has been realized between a color distribution features representing an image features which has an image structural feature and a frequency distribution features.
As described above, the conventional devices have the following problems.
First, since an image features indicative of feature of an image has various kinds, for the comparison and search of images, an image features of a kind common to both images to be compared (searched) should be prepared for the images.
In addition, even when an image features amount of a kind common to both the images is provided, a function of conducting comparison and search based on the image features should be further provided in an image search system. For the image search system to realize comparison and search based on various kinds of image features amounts such as an amount of distribution feature and a frequency distribution features, it should be provided with a similarity calculation means based on each kind of image features amount, resulting in making the device complicated and increasing manufacturing costs.
Secondly, conventional devices are incapable of comparing images whose features are represented by a color distribution features and a frequency distribution features, respectively. This is because alternate level conversion technique in practical use has not been realized between a color distribution features and a frequency distribution features.
For example, in a case where an image features of an inquiry image is a frequency distribution features, it is impossible to search a data base which records a color distribution features of an image to be searched to find an image similar to the inquiry image.